Dyadic distribution

A dyadic distribution is a specific type of discrete or categorical probability distribution that is of some theoretical importance in data compression.

Definition

A dyadic distribution on the nonnegative integers 0, 1, 2, ... is a probability distribution whose probability mass function is

$f(u) = 2^{-n_u},\quad u \in U$

where n_u is some (positive) integer. More generally it is a categorical distribution in which the probability assigned to any label is of the above form

It is possible to find a code defined on this distribution, which has an average code length that is equal to the entropy.

References

Cover, T.M., Joy A. Thomas, J.A. (2006) Elements of information theory, Wiley. ISBN 0471241954